In-field Plant Phenotyping using Model-free and Model-based methods

The ability to automatically correlate observable characteristics of plants to their underlying genetics will streamline selection methods in plant breeding which in-turn will help improve crop yields. Measurement of observable plant characteristics is called phenotyping. Currently established methods for phenotyping in the field are labour intensive, error prone and have limited throughput. This thesis investigates and develops automated computer vision based methods for doing in-field plant phenotyping on data collected using a robotic platform. In particular, we consider the problem of extracting phenotype values from a sequence of 2D images of a sorghum plant. We take the approach of performing multi-view 3D reconstruction on a sequence of 2D images in order to generate a 3D point cloud representation of the sorghum plant. We then use these point cloud reconstructions to perform point-level semantic segmentation followed by surface reconstruction to create 3D mesh models. We analyze the role of purely local geometric features in segmentation and the effect of addition of global context. Computational geometric methods are then employed for computing phenotype values like leaf length, leaf width, leaf area and stem diameter. We group these methods under model-free approaches for plant phenotyping. We also explore in parallel a model-based phenotype estimation approach. We formulate the model-based approach as generating parameterized 3D plant models and comparing those against the reference model whose phenotypes we wish to estimate. The parameters used for generating plant models are taken as random variables drawn from an underlying probability distribution. We then optimize for an objective of making the mass of this probability distribution approach the true parameters of the reference model. We compare the applicability and performance of the proposed model-free and model-based phenotyping approaches. These two approaches together lets us develop a framework for performing plant phenotype estimation on 3D plant models. We evaluate the qualitative and quantitative efficacy of our methods on data collected in both indoor greenhouse and outdoor field environments.